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FOREWORD 



This research was completed under Project 6323, Personnel Management Research 
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to Facilitate Personnel Research. 

This report has been reviewed and is approved. 

F.L. McLanathan, Lt Col, USAF 
Chief, Personnel Research Division 



ABSTRACT 



For the purposes of field testing the proposed Weighted Airman Promotion System, 
the Alaskan Air Command supplied data on 2,835 airmen eligible for promotion to grades 
E-4 through E-7 in the FY 69-B promotion cycle. Data included scores on the Specialty 
Knowledge Test and the Promotion Fitness Examination; points credited for time in 
grade, time in service, decorations, and Airman Performance Report mean overall 
evaluation; and ID information. Cases for which test scores were unavailable were 
eliminated; the final sample consisted of 2,290 airmen, 81 percent of the original 
eligibles. 

Average promotion board scores were computed by dividing each airman’s raw 
board score by the number of members on the promotion board and multiplying the 
quotient by 10. Weighted factors scores were computed by adding the factor scores. Two 
such total composite scores were obtained: a weighted factors score excluding the average 
board score and a weighted factors score including the average board score. The analyses 
concentrated on the comparisons of these two total composite scores. The hypothesis 
tested was that inclusion of the board score component in the weighted factors composite 
score does not have an effect on the ranking of the airmen. Ranks on the composite total 
scores with and without the board score were analyzed for strength of relationship. 
Average board scores were analyzed for differences in scoring across Air Force Bases and 
by different size board panels. 

The following results were obtained: (a) There were significant differences in 
average board score means between Elmendorf and Eielson AFBs for grade E-3 personnel. 
( [b ) There were significant differences in average board score means between 3-member 
and 5-member board panels, (c) There was a very high, near perfect relationship between 
weighted factors composites excluding and including the board score component, (cf) 
Rankings of individuals on the weighted factors composite were essentially unaffected by 
inclusion of the promotion board score. 

Since inclusion of a board score component had a negligible effect on the ranking 
by composite total score, and since means of board scores differed between bases and 
between boards with different panel sizes, it was concluded that including a board score 
factor merely inserted a non-visible component which would obscure explanations for 
non-promotion. 
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FIELD TEST OF THE WEIGHTED AIRMAN PROMOTION SYSTEM: 
PHASE I. ANALYSIS OF THE PROMOTION BOARD COMPONENT 
IN THE WEIGHTED FACTORS SYSTEM 



I. BACKGROUND 

As an outgrowth of the requirements for efficient, equitable promotion procedures which permit 
consideration of an optimum number of variables in the selection of airmen for promotion, a selection 
model has been designed and proposed for implementation. Part of the initial phase of this work, the field 
testing of the Weighted Airman Promotion System, is the subject of this report. 

Under existing procedures in the Air Force, airman promotions are carried out by means of 
promotion boards. These boards, consisting of three or five panel members, are convened for each 
promotion cycle to review a given number of folders of airmen who are eligible for promotion. Each board 
member assigns points for each airman (not less than 1 and not more than 10 points) to reflect his 
judgment about the merits of the performance documented in the airman’s folder. Promotion board scores, 
i.e., the sum of the scores for each airman given by these panel members, are reported to the major 
commands. The airmen are ranked from high to low on their board scores; ties in rankings are broken by 
crediting points for time in grade, time in service, Airman Performance Report rating, and decorations. 
Then selections for promotion are made according to quotas assigned for the different Air Force 
Specialties. Going down from high to low on the promotion board score ranking, the quota for each 
specialty is filled. 

During February and March 1968, research was conducted at the Personnel Research Division to 
develop a model for a new airman promotion system that could substitute for the present promotion 
system. Hopefully, the new system would provide more visibility to each airman in terms of numeric values 
on selected variables, which would in turn provide the airmen with an indication of how to increase his 
probability for promotion, or would give him evidence and information about the reasons he was not 
promoted. 

The brief description of this initial research which follows provides a background for the present 
study. A group consisting of 15 officers (7 Colonels, 2 Lt Colonels, and 6 Majors) and 16 noncommissioned 
officers (3 Senior Master Sergeants and 13 Chief Master Sergeants) served as teams to provide judgments of 
over two-thousand airmen who were eligible for promotion from grade E-5 to E-6. These selected judges 
were extremely well qualified, highly experienced personnel who were among the most competent persons 
in the Air Force to determine the relative merits of airmen eligible for promotion. The task of the Personnel 
Research Division was to capture the policy of these judges in ranking the individual airmen or, in other 
"words, to build a mathematical model identifying and reproducing as closely as possible the relative weights 
of the variables used by the judges in ranking the airmen for promotion. A model, derived from the 
selection policy of the largest, most consistent, and homogeneous group of judges, was recommended for 
adoption. This model with its weighting of selected variables gives the visibility, equity, and consistency 
over time desired in a system used to promote enlisted personnel. 

The merits and desirability of this new weighted factors promotion system were discussed at length 
during the USAF Airman Promotion Conference held in April 1968. The general consensus was that 
selection for promotion could be based either completely or to a great extent on the mathematical model 
consisting of the major selection factors, with little or no requirement for inclusion of a promotion board 
component as is currently used. The alternate solution of providing promotion boards with weights and 
factors to consider appeared to introduce excessive cost without enhancing the degree of visibility of the 
individuals under consideration. After more intensive discussion, the proposed weighted factors system 
developed by the Personnel Research Division, with factors and weights as proposed by the Division and 
modified by the panel of judges, was recommended for implementation at the earliest feasible date. 
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The weighted factors promotion system was approved by the Secretary of the Air Force on 
3 July 1968 and was subjected to a field test in the Alaskan Air Command. The purpose of the 
field test was to obtain tangible information with regard to the feasibility of the new system 
itself, to determine the relative contribution of a promotion board score to the weighted factors 
composite score, and to validate the Promotion Fitness Examination. The field test can thus be 
summarized under three major phases: 

1. Analysis of the effects of the promotion board score on predictive efficiency of the 
weighted factors composite score. 

2. Validation of the weighted Airman Promotion System. 

3. Validation of the Promotion Fitness Examination. 

This report is a summary of the first phase dealing with the effects of the promotion board score on 
the proposed weighted factors system. The second and third phases, i.e., validation of the weighted factors 
system itself and validation of the Promotion Fitness Examination, are reported separately because these 
phases attempt to answer entirely different kinds of questions. 
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II. EXPLANATION OF THE WEIGHTED FACTORS SYSTEM 

The components derived from the initial analysis of the selection policy of the panel of judges 
included scores for the Specialty Knowledge Test, the Promotion Fitness Examination, time in grade, time 
in service, decorations, and the mean overall evaluation from Airman Performance Reports, with a possible 
additional component — a promotion board score. Appendix I shows the factors and the maximum points 
for each, with an explanation of their use and computation for grades E4 through E-7. Also indicated are 
two alternative scoring procedures, one with the inclusion of a promotion board evaluation and the other 
with no promotion board component. 

It should be emphasized that the points indicated in Appendix I are maximum obtainable points. 
There is, of course, a very small probability that the maximum total points can be obtained by any one 
airman. In order to obtain the maximum number of points, an airman would have to obtain perfect scores 
on the factors involving tests (190 points), have a minimum of 20 years in service (40 points), have 10 years 
in grade (60 points), and have been awarded decorations for the maximum number of points (25 points: 
Medal of Honor, 15 points; Air Force Cross, 9 points; and one other decoration or some equivalent 
recognition for heroism, 1 point). The small probability of occurrence of a maximum total score limits the 
interpretation of these points in terms of percentage of contribution to the total score as shown in the 
appendix. Thus, no attempt should be made to assert that the score on the Specialty Knowledge Test, for 
instance, “contributes” 17 percent to the total score, or that the contribution of decorations is 5 percent. 
In fact, the percentage of contribution of an individual factor depends entirely on the individual scores on 
these factors in each airman’s case. The percentage; indicated in the appendix are valid only if the values for 
all factors are maximum. More explicitly, the percentages shown are not predetermined but, rather, are 
derived from the values in this particular example, showing in this case the relative contribution of each 
factor to the total score. 

Another point that should be kept in mind is that any weighted system is in a sense an arbitrary 
judgment of the group of people assigning the weights. In this instance, the weights derived and proposed 
were based on the captured policy of a team of Air Force managers. There is no guarantee that the policy 
of this group, or the weights arrived at by capturing their policy, would necessarily be strongly related to 
the judgment or policy of any one of the existing promotion boards. In fact, it is entirely possible that 
policies and weights used by different promotion boards would have differential values if analyzed 
mathematically. The aim of validation, however, will be to find an indicator expressing the degree of 
agreement between the lists of incumbents selected by a promotion board and those selected by the 
weighted factors system. Validation of the system is discussed in a separate report. 

In the present discussion, the contribution of the promotion board score to the weighted factors 
composite is considered, as well as the possibility and effects of eliminating this factor altogether. The 
nature of this discussion forces two basic assumptions: First, that the weighted factors promotion system is 
accepted for implementation; and second, that the weights used are accepted as proposed. The only topic 
addressed by this report is the question of retention or elimination of the promotion board component. 
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III. PROCEDURE 



Data Collection 

Data for the study were collected on personnel in the Alaskan Air Command. The data were compiled 
during the period 5 August 1968 to 1 October 1968 at Elmendorf and Eielson Air Force Bases for airmen 
in pay grades E-3 through E-6 who were eligible for promotion during the FY 1969-B promotion cycle. 
These cases did not include airmen at remote sites for whom no attempt was made to obtain Promotion 
Fitness Examination scores. Data included Specialty Knowledge Test scores, Airman Performance Report 
scores, Time-in-Grade scores, Time-in-Service scores, Decorations scores, and Promotion Board scores. The 
latter were given in raw score form; the maximum possible promotion board score was 30 for 3-member 
panels and 50 for 5-member panels. Elmendorf Air Force Base had both 3-member and 5-member panels, 
while Eielson used 5-member panels only. The score card work-deck format is shown as Appendix II. In 
addition to the factors described, the data cards included other dates and information, together with 
computed total composite score. This total score, however, did not include the Promotion Fitness 
Examination score. Further, the inclusion of a promotion board score with differing upper limits made it 
necessary to recompute a weighted factors composite score in which the board score component was 
adjusted for differences in panel size. Of the 2,835 cases for whom data were received, some were excluded 
either because no Specialty Knowledge Test scores were available or no Promotion Fitness Examination 
scores were available. Two other exclusions were caused by out-of-range data. Table 1 summarizes the 
sample used in the study. 



Table 1 . Study Sample by Pay Grade 

(N = 2,290) 



Cases 


E-3 


E-4 


E-5 


E-6 


Airman Eligible for Promotion 


605 


1,130 


619 


481 


Cases Excluded 


50 


280 


105 


110 


Total for Pay Grade 


555 


850 


514 


371 



Promotion Fitness Examination Score 

An experimental form of the Promotion Fitness Examination was administered to personnel in the 
Alaskan Air Command. Answer sheets from a total of 2,578 examinees were scored at the Personnel 
Research Division and were added to the information already received on those examinees from the 
Alaskan Air Command. 

Time-in-Grade and Time-in-Service Scores 

to assure maximum accuracy, Time-in-Grade and Time-in-Service scores were recomputed using Date 
of Rank (DOR) and Total Active Federal Military Service Date (TAFMSD) and the date 1 January 1969 for 
airmen in grade E-3 and 1 May 1969 for airmen in grades E-4 through E-6. Results were distributed against 
the Time-in-Grade and Time-in-Service scores supplied by the Alaskan Air Command. There was only one 
case for which the given Time-in-Service score differed from the computed one; this discrepancy, of course, 
was corrected. 

Average Promotion Board Score 

Before total composite weighted scores could be computed, a decision was necessary regarding use of 
the promotion board scores in their raw score form since the promotion panels were of two different sizes. 
Means and standard deviations were computed by grade, by Air Force base, and by number of panel 
members. These computations were used to determine whether scores would have to be standardized or 
whether some kind of average board score could be used with an adjustment for the difference in panel size. 



Table 2 summarizes the means, standard deviations, and number of cases in each category for the 
promotion board raw scores. It is apparent from the table that variances were similar enough to permit 
computation of an Average Board Score for each airman. This was done for each airman by dividing his 
score by the number of panel members and multiplying f his quotient by 10 to obtain a range between I 
and 100 as required and shown in Appendix 1 (Board Evaluation maximum points = 100). 

Table 3 summarizes the means and standard deviations of the computed Average Board Scores by 
grade, by Air Force base, and by number of panel members. A fairly sizable difference is indicated between 
the means of airmen in grade E-3 with 5-member panels (mean = 79.46) and 3-member panels (mean = 
63.67). This difference (79.46 - 63.67 ® 15.79) is statistically significant beyond the .0001 level of 
confidence. One might conclude, then, that a difference between the two means of these groups could 
happen less than once out of 10,000 times by chance alone. There appears to be a strong interaction 
between the size of the board panel and the mean score. A comparison between the means of grade E-3 
airmen at the two bases for groups having 5-member panels reflects a statistically significant difference 
beyond the .01 level of confidence. In other words, the difference between these means (79.46 for 
Elmendorf and 75.18 for Eielson) could happen only less than once out of 100 times by chance alone. 
Although no other significant differences are apparent from Tables 2 and 3, it is safe to conclude that 
promotion board scores in certain instances show significant differences when compared between different 
locations or even within the same location when the panel size is different. This conclusion cannot be 
generalized from the available data since this study deals with only two locations; however, there is a strong 
likelihood that one would find similar differences dealing with other promotion boards. 



Table 2. Promotion Board Raw Score Means, Standard Deviations, 
and Number of Cases by Grade, AFB, and Panel Size 



Grade 


Board 

Members 










Raw Scores 








Elmendorf 






Eielson 






Total 




N 


Mean 


SO 


N 


Mean 


SD 


N 


Mean 


SD 


E-3 


5 


38 


39.73 


5.17 


129 


37.59 


3.25 


167 


37.95 


4.23 




3 


388 


19.10 


3.49 


- 


• 


- 


388 


19.10 


3.49 


E-4 


5 


612 


36.87 


4.65 


238 


36.53 


4.53 


850 


36.77 


4.62 


E-5 


5 


406 


34.76 


4.00 


108 


34.75 


3.29 


514 


34.77 


3.70 


E-6 


5 


301 


37.94 


4.22 


70 


38.74 


3.66 


371 


38.08 


4.13 


Total 


3 


388 






• 






388 


19.10 


3.49 




5 


1,357 






545 






1,902 


36.59 


4.43 



Table 3. Average Board Score Means, Standard Deviations, and 
Number of Cases by Grade, AFB, and Panel Size 



Average Board Scores Times 10 



Grade 


Board 

Members 




Elmendorf 






Eielson 






Total 




N 


Mean 


SD 


N 


Mean 


SD 


N 


Maan 


SD 


E-3 


5 


38 


79.46 


10.34 


129 


75.18 


6.50 


167 


75.90 


8.46 




3 


388 


63.67 


11.63 


• 


- 


* 


388 


63.67 


11.63 


E-4 


5 


612 


73.74 


9.30 


238 


73.06 


9.06 


850 


73.54 


9.24 


E-5 


5 


406 


69.52 


8.00 


108 


69.50 


6.58 


514 


69.54 


7.40 


E-6 


5 


301 


75.88 


8.44 


70 


77.48 


7.32 


371 


76.16 


8.26 



Total 



388 

1,357 



388 

1,902 



63.67 

73.18 



11.63 

8.86 



545 



Weighted Factors Score 

The individual variables were distributed by grade. Weighted factors composite scores were computed 
two ways: First, the weighted factors composite score excluding the average promotion board score 
(Weighted Factors Score Without Average Board Score) and second, the weighted factors composite score 
including the average promotion board score (Weighted Factors Score With Average Board Score). 

Distribution of the Average Board Scores revealed, as expected, a high incidence of ties. Appendix III 
shows the number of ties and the number of actual promotions at the promotion quota cutoff points by 
grade. For instance, for grade E4, there were 21 2 promotions (quota cutoff point). Out of the total sample 
of airmen in grade E4 (850), there were 66 ties in Average Board Score for rank 212. Twenty-nine of 
these 66 ties were actually promoted; 37 of the ties were not promoted. 

The high incidence of ties reflects the limited discrimination power of board scores when grades are 
analyzed without further breakdown by Control Air Force Specialty Codes (CAFSC). However, even 
though the number of ties within a particular CAFSC may be smaller, the probability of ties in board scores 
is still much larger than the probability of ties in weighted factors total scores. The implication is that there 
is a better discrimination using the Weighted Factors Score With Average Board Score than there is using 
the Average Board Score alone. One could argue that in the case of a tie, the board “takes another look” at 
the individuals* records in order to break the tie. However, one could also argue that if this “other look” 
differentiates the individuals, then either the “first look” was erroneous (that is, there should not have been 
a tie), or this “second look** is a factor which is invisible to the airman and is not detectable or measurable 
by any particular mathematical model. 



IV. EFFECTS OF THE AVERAGE BOARD SCORE ON THE WEIGHTED FACTORS SCORE 

Results Obtained for Total Sample with a Maximum Board Score Component of 100 

The basic question of this phase of the investigation is: What effect, if any, does inclusion of the 
promotion board component in the weighted factors system have on the ranking of individuals? 

To answer this question, each airman within each grade was ranked on three variables: (a) Weighted 
Factors Score Without Average Board Score, (b) Weighted Factors Score With Average Board Score, and (c) 
Average Board Score. Analyses were performed at different hypothetical promotion percentage quotas of 
20, 30, 40, and 50 percent. That is, airmen were ranked by Weighted Factors Score Without Average Board 
Score, and the first 20 percent of the particular grade (1 14 for E-3, for example) were analyzed in terms of 
matches between these ranks and ranks on the Weighted Factors Score With Average Board Score. These 
percentages were computed as +1.5 percent to accommodate for ties. For example, for grade C-3, 20 
percent of 555 is 111. Because of ties, the quota number 114 was chosen instead. Of these 114 airmen, 
there were 102 whose ranks on the Weighted Factors Score Without Average Board Score and on the Weighted 
Factors Score With Average Board Score were both better than 1 14. The ratio of 102 to 1 14 (89.5 percent) ' 
represents the overlap between the two ranks. This ratio indicates the effects of adding the board score 
component to the weighted factors score. That is, 89.5 percent of the individuals who would have been 
promoted using the Weighted Factors Score Without Average Board Score would have also been promoted 
by the Weighted Factors Score With Average Board Score. The rank overlap and overlap percent (or index) 
is defined in this study as the number of the corresponding percentage of individuals who could be 
promoted by both promotion systems when the two systems are compared, or the number and percentage 
of individuals who would actually have been promoted by a system when this system is compared with 
actual promotions. 

Table 4 summarizes these rank overlaps by grade. The table also includes the overlap at the actual 
promotion quota cutoff point (478 for grade E-3, for example). It can be seen from the table that the 
lowest overlap index is 87 percent, but that the majority of these percentages are at the 90 percent level or 
higher. This implies that the ranking of individuals in terms of the weighted factors Score Without Average 
Board Score changed very little when ranking was in terms of the Weighted Factors Score With Average 
Board Score. 



Table 4 . Overlap Between Ranks on Weighted Factors Scores With and 
Without Average Board Score 



Hypothetical 

Promotion 

Quota 


Promotion 
Quota N 


Number 
Ranks on 
Both Scores 
Exceeding 
Quota Cutoff 


Percent 

Agreement 

Between 

Ranks 


Promotion 
Quota N 


Number 
Ranks on 
Both Scores 
Exceeding 
Quota Cutoff 


Percent 

Agreement 

Between 

Ranks 






Grade E-3 






Grade E4 




20% 


114 


102 


89.5 


170 


153 


90.0 


30% 


166 


156 


94.0 


255 


230 


90.2 


40% 


222 


205 


92.3 


341 


323 


94.7 


50% 


269 


249 


92.6 


422 


400 


94.8 


Actual 














Promotions 


478 


467 


97.7 


212 


192 


90.6 






Grade. E-5 






Grade E-6 




20% 


105 


100 


95.2 


77 


69 


89.6 


30% 


154 


147' 


95.5 


110 


98 


89.1 


40% 


207 


201 


97.1 


149 


139 


93.3 


50% 


257 


251 


97.7 


189 


177 


94.7 


Actual 














Promotions 


89 


81 


91.0 


46 


40 


87.0 



Table 5 summarizes the intercorrelations between the Weighted Factors Score Without Average Board 
Score and the Weighted Factors Score With Average Board Score. From the table, it is clear that there is a 
strong relationship between the two variables. In fact, the correlations are near perfect. Again, it is clear 
that adding the board score component to the weighted factors score has very little, if any, effect upon the 
ranking of the individuals or upon their total weighted factors score. In other words, adding the board score 
has the effect of adding a near constant to every score, thus increasing the total score but changing the 
ranking very little. Table 6 summarizes the intercorrelations between rankings on the Weighted Factors 
Score Without Average Board Score and the Weighted Factors Score With Average Board Score. 



Table 5. Intercorrelations Between 
Weighted Factors Scores With and 
Without Average Board Score 



Grade 


Intarcor relation 
Between 
Variables 


E-3 


.9741 


E-4 


.9814 


E-5 


.9857 


E-6 


.9826 



Table 6. Intercorrelations Between 
Ranks on Weighted Factors Scores 
With and Without Average Board Score 


Grade 


Intercorrelatlon 

Between 

Variables 


E-3 


.9733 


E-4 


.9801 


E-5 


.9871 


E-6 


.9871 



0 




Results Obtained for Total Sample with a Maximum Board Score Component of 200 

A maximum of 100 points for the proposed board score component for the weighted factors system 
was specified as the value to be investigated in this field test. The assignment of this maximum value was 
made independently from the assignment of values for the other factors in the construction of the 
weighting system. Since a board score component was not a factor in the policy capturing exercises which 
yielded the weights for the other factors, the assignment of a 100-point maximum was, in a sense, a rather 
arbitrary decision. Before the results of this field test became available, it was impossible to predict with 
any assurance whether the board score component would significantly affect the ranking of incumbents for 
promotion as determined by a weighted factors system. Results of the preceding analysis indicated that a 
100-point maximum board score component introduces virtually no change in the ranking of individuals 
based on a weighted factors score which includes no board score component. 

In an effort to determine whether a larger board score component would produce significant changes 
in the ranking of individuals, the size of the board score component was doubled, and the relationships 
were studied between the Weighted Factors Score Without Average Board Score and a score with the board 
score component increased, the Weighted Factors Score With Accentuated Average Board Score. The 
maximum board score for this trial system was 200 points. Ranks on the Weighted Factors Score Without 
Average Board Score were compared with ranks on the Weighted Factors Score With Accentuated Average 
Board Score. These comparisons, together with comparisons of ranks obtained when the 100-point 
maximum board score was used, are presented in Table 7. 

The values in Table 7 imply that doubling the board score component does not, in general, decrease 
the overlap between Weighted Factors Scores With or Without Average Board Score. For example, 96.2 
percent of grade E-4 personnel who would have been promoted by the weighted factors system excluding 
the board score component would also have been promoted by the weighted factors system using the 
200-point maximum board score component. In the same grade, this percentage was 90.1 percent when the 
100-point maximum board score component was used. Doubling the size of the board score component 
does not appear to decrease the number of matches (overlap ranks), or the number of individuals who 
would be promoted by both systems (Weighted Factors Scores With and Without Average Board Score). It 
seems apparent, then, that even with its value doubled, the inclusion of a board score component in the 
weighted factors composite system fails to influence the ranking of airmen. 



Table 7. Overlap Between Ranks on Weighted Factors Scores Without Average Board Score 
vs. Weighted Factors Scores with Average Board Score (Using 100-Point Maximum) 
and With Accentuated Average Board Score (Using 200-Point Maximum) 



Weighted Factors Scores 
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Promotion 

Quota 

Cutoff 


Number 
Ranks on 
Both Scores 
Exceeding 
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Percent 
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Ranks 


Promotion 

Quota 

Cutoff 


Number 
Ranks on 
Both Scores 
Exceeding 
Quota Cutoff 


Percent 

Agreement 

Between 

Ranks 


Without Board Score vs . 




Grade E-3 






Grade E-4 




478 


429 


89.8 


212 


204 


96.2 


With Accentuated Board Score 


Without Board Score vs. 
With Board Score 


478 


467 


97.7 


212 


197 


90.1 






Grade E-5 






Grade E-6 




Without Board Score vs. 

With Accentuated Board Score 


89 


84 


94.4 


46 


46 


100.0 


Without Board Score vs. 
With Board Score 


89 


81 


91.0 


46 


40 


87.0 



Results Obtained for Selected C AFSCs 

The sample from the Alaskan Air Command ( N = 2,290) did not lend itself to complete analysis by 
CAFSC. Most specialties had small frequencies of eligible airmen. It was possible, however, to select a few 
CAFSCs with enough airmen who were eligible for promotion to conduct a meaningful mathematical 
analysis. Table 8 shows the number of eligible airmen by grade and by CAFSC. Some of the frequencies 
were rather small due to the attempt to analyze the same CAFSCs across the four grades. 

Essentially the same type of analysis was performed on these selected CAFSC groups as was carried 
out in the case of the total grades. For each CAFSC group, the Weighted Factors Scores Without Average 
Board Score were correlated with their Weighted Factors Scores With Average Board Score. These 
correlations, also shown in Table 8 by grade and by the selected CAFSCs, serve as indices of the effect of 
adding the board score component to the weighted factors score. As expected, the analysis of the weighted 
factors scores with and without the board score showed an even more marked relationship when broken 
down into CAFSCs. The near perfect correlations ranged from .9726 to .9941. The addition of the board 
score component seems to have little or no effect on the relative ranking of the individuals on the weighted 
factors score. 



Table 8. Intercorrelations Between Weighted Factors Scores With and Without Average 
Board Score by Selected CAFSCs and by Grade 
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E-3 


49 


.9768 


23 


.9836 


41 


.9726 


32 


.9806 


25 


.9789 


108 


.9770 


E-4 


55 


.9815 


74 


.9926 


57 


.9869 


35 


.9854 


50 


.9870 


35 


.9914 


E-5 


34 


.9879 


42 


.9838 


44 


.9851 


10 


.9941 


42 


.9675 


25 


.9843 


E-6 


31 


.9872 


17 


.9927 


25 


.9870 


11 


.9846 


33 


.9759 


13 


.9746 



V. SUMMARY AND CONCLUSIONS 

A field test of the weighted factors promotion system was conducted using data furnished by the 
Alaskan Air Command. The sample included 2,290 airmen in pay grades E-3 through E-6 who were eligible 
for promotion. The analysis of this report concentrated on the contribution of the promotion board 
component to the weighted factors system and its effects on the relative ranking of the individuals. 

Promotion board scores were converted to a scale with 100 as the upper limit. Weighted factors scores 
were computed with and without the average board scores. Factors of the weighted factors system were 
distributed and analyzed. The airmen in the study were ranked on both total scores, i.e., total weighted 
factors scores excluding the average board score and total weighted factors scores including the average 
board score. Overlaps in ranks on these two total scores were computed for various promotion percentage 
quotas and for the actual number of promoted individuals within each grade. The alternative total scores 
and ranks on these scores were also analyzed in terms of correlations. Separate and additional analyses were 
performed for selected CAFSCs where there were sufficient numbers of airmen eligible for promotion to 
permit meaningful analyses. 

The following results were found from the analyses performed: 

1. For airmen in grade E-3, promotion board scores differed significantly for 3-member and 
5-member panels at Elmendorf Air Force Base. 

2. Mean board scores at Elmendorf and Eielson Air Force Bases differed significantly for grade E-3 
airmen, suggesting that board scores may in part be dependent on base of assignment. 



3. Inclusion of the average promotion board score in the weighted factors score had little or no effect 
on the relative ranking of the individuals on the total weighted factors score. This was true for each grade 
and each selected CAFSC analyzed. 

4. Giving more weight to the board score (i.e., doubling its value) did not change the relative ranking 
of individuals. Inclusion of this hypothetical “heavy” board score left rankings of individuals on' the 
composite score practically unaffected. 

The following conclusions are based on the two fundamental assumptions stated earlier, i.e., that the 
weighted factors promotion system is accepted as the new system for promotion and that the weights 
assigned to the factors are accepted as valid. In this light, there is conclusive evidence that addition of a 
promotion board score to the weighted factors score is unnecessary. Analyses of this study clearly show 
that addition of the board score to the composite score changes very little, if any, the ranking of the 
individuals on that composite score. In addition, airmen cannot interpret board scores since the 
components of board scores are not visible to them. Thus, under the assumptions of this study, the 
inclusion of a board score component appears to be unnecessary, as well as inconsistent with the visibility 
objective of the proposed weighted factors promotion system. 



APPENDfX /: AIRMAN PROMOTION SELECTION FACTORS AND POINTS 

Grades E-4 through E-7 



Computation With Board Score Computation Without Board Score 



Maximum Maximum 

Selection Factor Points Percentage Points Percentage 



Specialty Knowledge Test (SKT) Score 


95 


17 


95 


21 


Promotion Fitness Examination (PFE) Score 


95 


17 


95 


21 


Time-In-Service (TIS) Score 


40 


7 


40 


9 


Time-In-Grade (TIG) Score 


60 


11 


60 


13 


Decoration Score 


25 


5 


25 


6 


Airman Performance Reports (APR) 


135 


25 


135 


30 


Board Evaluation 


100 


18 








550 


100 


450 


100 



Explanation of Factors 

The SKT and PFE will be administered annually. 

Points for the SKT and PFE scores will be actual percentile scores obtained in these tests (in 5-point 
increments). 

Time-in-Service will be computed by multiplying TAFMS by 2. Less than 6 months will count as 1 point; 
over 6 months will count a full year, 2 points. A cutoff score of 40 points, for 20 years TAFMS, has been 
established. 

Time-in-Grade will be computed at the rate of Vi point per month up to a maximum of 120 months, 60 
points; 15 days or less will be dropped, 16 or more will count as a full month. 

Decorations will be assigned points according to their order of precedence. The maximum number of points 
attainable is 25. Decorations will count for promotion regardless of the military service in which they were 
earned. 

The Airman Performance Report score is obtained by multiplying the overall evaluation mean by 15. The 
mean is based on reports for a 5-year period prior to the eligibility date, not to exceed ten reports. 

The Board Score will be based on a review by the board that concentrates on those items not previously 
weighted; e.g., education level and efforts to improve self in terms of formal education, technical 
knowledge, etc. Reduced selection folder will consist of Category A favorable communications, APR word 
picture, and pages 2 and 4 of the Air Force Form 7. 



APPENDIX II. SCORE CARD WORK-DECK FORMAT 



DATA 



CARD COLUMNS 



Not Used 

Air Force Service Number (AFSN) 

Number of Board Members 
Name 

Date of Rank (DOR) 

Total Active Federal Military Service Date (TAFMSD) 
Control Air Force Specialty Code (CAFSC) 

Board Score 

Airman Performance Report (APR) Score 
Decoration Score 

Specialty Knowledge Test (SKT) Score 
Time-in-Grade (TIG) Score 
Time-in-Service (TIS) Score 
Not Used 
Total Score 
' Not Used 
PAS 

Not Used 



1 

4-11 

12 

13-30 

31-36 

3742 

4349 

50-52 

53-55 

56-57 

58-59 

60-61 

62-63 

64-65 

66-69 

70-73 

74-79 

80 



19 



Number of Ties 



APPENDIX III. TIED AVERAGE BOARD SCORES AT PROMOTION QUOTA CUTOFF 
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46 



